For Multi-objective optimization (MOO) in the AxClient, objectives are specified through the ObjectiveProperties dataclass. An ObjectiveProperties requires a boolean minimize, and also accepts an optional floating point threshold. If a threshold is not specified, Ax will infer it through the use of heuristics. If the user knows the region of interest (because they have specs or prior knowledge), then specifying the thresholds is preferable to inferring it. But if the user would need to guess, inferring is preferable.
To learn more about how to choose a threshold, see Set Objective Thresholds to focus candidate generation in a region of interest. See the Service API Tutorial for more infomation on running experiments with the Service API.
from ax.service.ax_client import AxClient
from ax.service.utils.instantiation import ObjectiveProperties
import torch
# Plotting imports and initialization
from ax.utils.notebook.plotting import render, init_notebook_plotting
from ax.plot.pareto_utils import compute_posterior_pareto_frontier
from ax.plot.pareto_frontier import plot_pareto_frontier
init_notebook_plotting()
# Load our sample 2-objective problem
from botorch.test_functions.multi_objective import BraninCurrin
branin_currin = BraninCurrin(negate=True).to(
dtype=torch.double,
device= torch.device("cuda" if torch.cuda.is_available() else "cpu"),
)
[INFO 08-19 17:52:23] ax.utils.notebook.plotting: Injecting Plotly library into cell. Do not overwrite or delete cell.
ax_client = AxClient()
ax_client.create_experiment(
name="moo_experiment",
parameters=[
{
"name": f"x{i+1}",
"type": "range",
"bounds": [0.0, 1.0],
}
for i in range(2)
],
objectives={
# `threshold` arguments are optional
"a": ObjectiveProperties(minimize=False, threshold=branin_currin.ref_point[0]),
"b": ObjectiveProperties(minimize=False, threshold=branin_currin.ref_point[1])
},
overwrite_existing_experiment=True,
is_test=True,
)
[INFO 08-19 17:52:23] ax.service.ax_client: Starting optimization with verbose logging. To disable logging, set the `verbose_logging` argument to `False`. Note that float values in the logs are rounded to 2 decimal points.
[INFO 08-19 17:52:23] ax.service.utils.instantiation: Inferred value type of ParameterType.FLOAT for parameter x1. If that is not the expected value type, you can explicity specify 'value_type' ('int', 'float', 'bool' or 'str') in parameter dict.
[INFO 08-19 17:52:23] ax.service.utils.instantiation: Inferred value type of ParameterType.FLOAT for parameter x2. If that is not the expected value type, you can explicity specify 'value_type' ('int', 'float', 'bool' or 'str') in parameter dict.
[INFO 08-19 17:52:23] ax.core.experiment: The is_test flag has been set to True. This flag is meant purely for development and integration testing purposes. If you are running a live experiment, please set this flag to False
[INFO 08-19 17:52:23] ax.modelbridge.dispatch_utils: Using Bayesian optimization since there are more continuous parameters than there are categories for the unordered categorical parameters.
[INFO 08-19 17:52:23] ax.modelbridge.dispatch_utils: Using Bayesian Optimization generation strategy: GenerationStrategy(name='Sobol+MOO', steps=[Sobol for 5 trials, MOO for subsequent trials]). Iterations after 5 will take longer to generate due to model-fitting.
In the case of MOO experiments, evaluation functions can be any arbitrary function that takes in a dict of parameter names mapped to values and returns a dict of objective names mapped to a tuple of mean and SEM values.
def evaluate(parameters):
evaluation = branin_currin(torch.tensor([parameters.get("x1"), parameters.get("x2")]))
# In our case, standard error is 0, since we are computing a synthetic function.
# Set standard error to None if the noise level is unknown.
return {"a": (evaluation[0].item(), 0.0), "b": (evaluation[1].item(), 0.0)}
for i in range(25):
parameters, trial_index = ax_client.get_next_trial()
# Local evaluation here can be replaced with deployment to external system.
ax_client.complete_trial(trial_index=trial_index, raw_data=evaluate(parameters))
[INFO 08-19 17:52:24] ax.service.ax_client: Generated new trial 0 with parameters {'x1': 0.95, 'x2': 0.4}.
[INFO 08-19 17:52:24] ax.service.ax_client: Completed trial 0 with data: {'a': (-14.39, 0.0), 'b': (-7.28, 0.0)}.
[INFO 08-19 17:52:24] ax.service.ax_client: Generated new trial 1 with parameters {'x1': 0.61, 'x2': 0.6}.
[INFO 08-19 17:52:24] ax.service.ax_client: Completed trial 1 with data: {'a': (-59.08, 0.0), 'b': (-6.28, 0.0)}.
[INFO 08-19 17:52:24] ax.service.ax_client: Generated new trial 2 with parameters {'x1': 0.58, 'x2': 0.59}.
[INFO 08-19 17:52:24] ax.service.ax_client: Completed trial 2 with data: {'a': (-49.2, 0.0), 'b': (-6.46, 0.0)}.
[INFO 08-19 17:52:24] ax.service.ax_client: Generated new trial 3 with parameters {'x1': 0.95, 'x2': 0.92}.
[INFO 08-19 17:52:24] ax.service.ax_client: Completed trial 3 with data: {'a': (-131.57, 0.0), 'b': (-4.3, 0.0)}.
[INFO 08-19 17:52:24] ax.service.ax_client: Generated new trial 4 with parameters {'x1': 0.48, 'x2': 0.72}.
[INFO 08-19 17:52:24] ax.service.ax_client: Completed trial 4 with data: {'a': (-63.77, 0.0), 'b': (-5.92, 0.0)}.
[INFO 08-19 17:52:24] ax.service.ax_client: Generated new trial 5 with parameters {'x1': 0.0, 'x2': 0.0}.
[INFO 08-19 17:52:24] ax.service.ax_client: Completed trial 5 with data: {'a': (-308.13, 0.0), 'b': (-3.0, 0.0)}.
[INFO 08-19 17:52:24] ax.service.ax_client: Generated new trial 6 with parameters {'x1': 1.0, 'x2': 0.0}.
[INFO 08-19 17:52:24] ax.service.ax_client: Completed trial 6 with data: {'a': (-10.96, 0.0), 'b': (-10.18, 0.0)}.
[INFO 08-19 17:52:25] ax.service.ax_client: Generated new trial 7 with parameters {'x1': 0.0, 'x2': 1.0}.
[INFO 08-19 17:52:25] ax.service.ax_client: Completed trial 7 with data: {'a': (-17.51, 0.0), 'b': (-1.18, 0.0)}.
[INFO 08-19 17:52:26] ax.service.ax_client: Generated new trial 8 with parameters {'x1': 0.0, 'x2': 0.79}.
[INFO 08-19 17:52:26] ax.service.ax_client: Completed trial 8 with data: {'a': (-41.76, 0.0), 'b': (-1.41, 0.0)}.
[INFO 08-19 17:52:27] ax.service.ax_client: Generated new trial 9 with parameters {'x1': 0.14, 'x2': 1.0}.
[INFO 08-19 17:52:27] ax.service.ax_client: Completed trial 9 with data: {'a': (-10.72, 0.0), 'b': (-5.06, 0.0)}.
[INFO 08-19 17:52:28] ax.service.ax_client: Generated new trial 10 with parameters {'x1': 0.06, 'x2': 0.95}.
[INFO 08-19 17:52:28] ax.service.ax_client: Completed trial 10 with data: {'a': (-5.44, 0.0), 'b': (-3.44, 0.0)}.
[INFO 08-19 17:52:29] ax.service.ax_client: Generated new trial 11 with parameters {'x1': 0.73, 'x2': 0.18}.
[INFO 08-19 17:52:29] ax.service.ax_client: Completed trial 11 with data: {'a': (-21.47, 0.0), 'b': (-9.99, 0.0)}.
[INFO 08-19 17:52:31] ax.service.ax_client: Generated new trial 12 with parameters {'x1': 0.02, 'x2': 0.95}.
[INFO 08-19 17:52:31] ax.service.ax_client: Completed trial 12 with data: {'a': (-14.71, 0.0), 'b': (-2.04, 0.0)}.
[INFO 08-19 17:52:34] ax.service.ax_client: Generated new trial 13 with parameters {'x1': 0.04, 'x2': 1.0}.
[INFO 08-19 17:52:34] ax.service.ax_client: Completed trial 13 with data: {'a': (-7.45, 0.0), 'b': (-2.73, 0.0)}.
[INFO 08-19 17:52:37] ax.service.ax_client: Generated new trial 14 with parameters {'x1': 0.08, 'x2': 0.96}.
[INFO 08-19 17:52:37] ax.service.ax_client: Completed trial 14 with data: {'a': (-2.65, 0.0), 'b': (-4.14, 0.0)}.
[INFO 08-19 17:52:38] ax.service.ax_client: Generated new trial 15 with parameters {'x1': 0.11, 'x2': 0.87}.
[INFO 08-19 17:52:38] ax.service.ax_client: Completed trial 15 with data: {'a': (-0.72, 0.0), 'b': (-5.14, 0.0)}.
[INFO 08-19 17:52:41] ax.service.ax_client: Generated new trial 16 with parameters {'x1': 0.02, 'x2': 1.0}.
[INFO 08-19 17:52:41] ax.service.ax_client: Completed trial 16 with data: {'a': (-12.14, 0.0), 'b': (-1.92, 0.0)}.
[INFO 08-19 17:52:43] ax.service.ax_client: Generated new trial 17 with parameters {'x1': 0.06, 'x2': 0.98}.
[INFO 08-19 17:52:43] ax.service.ax_client: Completed trial 17 with data: {'a': (-3.95, 0.0), 'b': (-3.65, 0.0)}.
[INFO 08-19 17:52:45] ax.service.ax_client: Generated new trial 18 with parameters {'x1': 0.09, 'x2': 0.92}.
[INFO 08-19 17:52:45] ax.service.ax_client: Completed trial 18 with data: {'a': (-1.58, 0.0), 'b': (-4.6, 0.0)}.
[INFO 08-19 17:52:49] ax.service.ax_client: Generated new trial 19 with parameters {'x1': 0.01, 'x2': 1.0}.
[INFO 08-19 17:52:49] ax.service.ax_client: Completed trial 19 with data: {'a': (-14.75, 0.0), 'b': (-1.54, 0.0)}.
[INFO 08-19 17:52:53] ax.service.ax_client: Generated new trial 20 with parameters {'x1': 0.03, 'x2': 1.0}.
[INFO 08-19 17:52:53] ax.service.ax_client: Completed trial 20 with data: {'a': (-9.71, 0.0), 'b': (-2.31, 0.0)}.
[INFO 08-19 17:52:55] ax.service.ax_client: Generated new trial 21 with parameters {'x1': 0.0, 'x2': 1.0}.
[INFO 08-19 17:52:56] ax.service.ax_client: Completed trial 21 with data: {'a': (-16.12, 0.0), 'b': (-1.36, 0.0)}.
[INFO 08-19 17:52:58] ax.service.ax_client: Generated new trial 22 with parameters {'x1': 0.05, 'x2': 1.0}.
[INFO 08-19 17:52:58] ax.service.ax_client: Completed trial 22 with data: {'a': (-5.9, 0.0), 'b': (-3.07, 0.0)}.
[INFO 08-19 17:53:01] ax.service.ax_client: Generated new trial 23 with parameters {'x1': 0.01, 'x2': 1.0}.
[INFO 08-19 17:53:01] ax.service.ax_client: Completed trial 23 with data: {'a': (-13.43, 0.0), 'b': (-1.73, 0.0)}.
[INFO 08-19 17:53:04] ax.service.ax_client: Generated new trial 24 with parameters {'x1': 0.06, 'x2': 1.0}.
[INFO 08-19 17:53:04] ax.service.ax_client: Completed trial 24 with data: {'a': (-4.88, 0.0), 'b': (-3.34, 0.0)}.
objectives = ax_client.experiment.optimization_config.objective.objectives
frontier = compute_posterior_pareto_frontier(
experiment=ax_client.experiment,
data=ax_client.experiment.fetch_data(),
primary_objective=objectives[1].metric,
secondary_objective=objectives[0].metric,
absolute_metrics=["a", "b"],
num_points=20,
)
render(plot_pareto_frontier(frontier, CI_level=0.90))
In the rest of this tutorial, we will show two algorithms available in Ax for multi-objective optimization and visualize how they compare to eachother and to quasirandom search.
MOO covers the case where we care about multiple
outcomes in our experiment but we do not know before hand a specific weighting of those
objectives (covered by ScalarizedObjective) or a specific constraint on one objective
(covered by OutcomeConstraints) that will produce the best result.
The solution in this case is to find a whole Pareto frontier, a surface in outcome-space containing points that can't be improved on in every outcome. This shows us the tradeoffs between objectives that we can choose to make.
Optimize a list of M objective functions $ \bigl(f^{(1)}( x),..., f^{(M)}( x) \bigr)$ over a bounded search space $\mathcal X \subset \mathbb R^d$.
We assume $f^{(i)}$ are expensive-to-evaluate black-box functions with no known analytical expression, and no observed gradients. For instance, a machine learning model where we're interested in maximizing accuracy and minimizing inference time, with $\mathcal X$ the set of possible configuration spaces
In a multi-objective optimization problem, there typically is no single best solution. Rather, the goal is to identify the set of Pareto optimal solutions such that any improvement in one objective means deteriorating another. Provided with the Pareto set, decision-makers can select an objective trade-off according to their preferences. In the plot below, the red dots are the Pareto optimal solutions (assuming both objectives are to be minimized).
Given a reference point $ r \in \mathbb R^M$, which we represent as a list of M ObjectiveThresholds, one for each coordinate, the hypervolume (HV) of a Pareto set $\mathcal P = \{ y_i\}_{i=1}^{|\mathcal P|}$ is the volume of the space dominated (superior in every one of our M objectives) by $\mathcal P$ and bounded from above by a point $ r$. The reference point should be set to be slightly worse (10% is reasonable) than the worst value of each objective that a decision maker would tolerate. In the figure below, the grey area is the hypervolume in this 2-objective problem.
The below plots show three different sets of points generated by the qEVHI algorithm with different objective thresholds (aka reference points). Note that here we use absolute thresholds, but thresholds can also be relative to a status_quo arm.
The first plot shows the points without the ObjectiveThresholds visible (they're set far below the origin of the graph).
The second shows the points generated with (-18, -6) as thresholds. The regions violating the thresholds are greyed out. Only the white region in the upper right exceeds both threshold, points in this region dominate the intersection of these thresholds (this intersection is the reference point). Only points in this region contribute to the hypervolume objective. A few exploration points are not in the valid region, but almost all the rest of the points are.
The third shows points generated with a very strict pair of thresholds, (-18, -2). Only the white region in the upper right exceeds both thresholds. Many points do not lie in the dominating region, but there are still more focused there than in the second examples.
A deeper explanation of our the qEHVI and qParEGO algorithms this notebook explores can be found at https://arxiv.org/abs/2006.05078, and the underlying BoTorch implementation has a researcher-oriented tutorial at https://botorch.org/tutorials/multi_objective_bo.
import pandas as pd
from ax import *
import numpy as np
from ax.metrics.noisy_function import NoisyFunctionMetric
from ax.service.utils.report_utils import exp_to_df
from ax.runners.synthetic import SyntheticRunner
# Factory methods for creating multi-objective optimization modesl.
from ax.modelbridge.factory import get_MOO_EHVI, get_MOO_PAREGO
# Analysis utilities, including a method to evaluate hypervolumes
from ax.modelbridge.modelbridge_utils import observed_hypervolume
x1 = RangeParameter(name="x1", lower=0, upper=1, parameter_type=ParameterType.FLOAT)
x2 = RangeParameter(name="x2", lower=0, upper=1, parameter_type=ParameterType.FLOAT)
search_space = SearchSpace(
parameters=[x1, x2],
)
To optimize multiple objective we must create a MultiObjective containing the metrics we'll optimize and MultiObjectiveOptimizationConfig (which contains ObjectiveThresholds) instead of our more typical Objective and OptimizationConfig
We define NoisyFunctionMetrics to wrap our synthetic Branin-Currin problem's outputs. Add noise to see how robust our different optimization algorithms are.
class MetricA(NoisyFunctionMetric):
def f(self, x: np.ndarray) -> float:
return float(branin_currin(torch.tensor(x))[0])
class MetricB(NoisyFunctionMetric):
def f(self, x: np.ndarray) -> float:
return float(branin_currin(torch.tensor(x))[1])
metric_a = MetricA("a", ["x1", "x2"], noise_sd=0.0, lower_is_better=False)
metric_b = MetricB("b", ["x1", "x2"], noise_sd=0.0, lower_is_better=False)
mo = MultiObjective(
objectives=[Objective(metric=metric_a), Objective(metric=metric_b)],
)
objective_thresholds = [
ObjectiveThreshold(metric=metric, bound=val, relative=False)
for metric, val in zip(mo.metrics, branin_currin.ref_point)
]
optimization_config = MultiObjectiveOptimizationConfig(
objective=mo,
objective_thresholds=objective_thresholds,
)
These construct our experiment, then initialize with Sobol points before we fit a Gaussian Process model to those initial points.
# Reasonable defaults for number of quasi-random initialization points and for subsequent model-generated trials.
N_INIT = 6
N_BATCH = 25
def build_experiment():
experiment = Experiment(
name="pareto_experiment",
search_space=search_space,
optimization_config=optimization_config,
runner=SyntheticRunner(),
)
return experiment
## Initialize with Sobol samples
def initialize_experiment(experiment):
sobol = Models.SOBOL(search_space=experiment.search_space)
for _ in range(N_INIT):
experiment.new_trial(sobol.gen(1)).run()
return experiment.fetch_data()
We use quasirandom points as a fast baseline for evaluating the quality of our multi-objective optimization algorithms.
sobol_experiment = build_experiment()
sobol_data = initialize_experiment(sobol_experiment)
sobol_model = Models.SOBOL(
experiment=sobol_experiment,
data=sobol_data,
)
sobol_hv_list = []
for i in range(N_BATCH):
generator_run = sobol_model.gen(1)
trial = sobol_experiment.new_trial(generator_run=generator_run)
trial.run()
exp_df = exp_to_df(sobol_experiment)
outcomes = np.array(exp_df[['a', 'b']], dtype=np.double)
# Fit a GP-based model in order to calculate hypervolume.
# We will not use this model to generate new points.
dummy_model = get_MOO_EHVI(
experiment=sobol_experiment,
data=sobol_experiment.fetch_data(),
)
try:
hv = observed_hypervolume(modelbridge=dummy_model)
except:
hv = 0
print("Failed to compute hv")
sobol_hv_list.append(hv)
print(f"Iteration: {i}, HV: {hv}")
sobol_outcomes = np.array(exp_to_df(sobol_experiment)[['a', 'b']], dtype=np.double)
Iteration: 0, HV: 0.0 Iteration: 1, HV: 0.0 Iteration: 2, HV: 0.0 Iteration: 3, HV: 0.0 Iteration: 4, HV: 0.0 Iteration: 5, HV: 0.0 Iteration: 6, HV: 0.0 Iteration: 7, HV: 0.0 Iteration: 8, HV: 0.0 Iteration: 9, HV: 0.0 Iteration: 10, HV: 0.0 Iteration: 11, HV: 0.0 Iteration: 12, HV: 0.0 Iteration: 13, HV: 0.0 Iteration: 14, HV: 0.0 Iteration: 15, HV: 0.0 Iteration: 16, HV: 0.0 Iteration: 17, HV: 0.0 Iteration: 18, HV: 0.0 Iteration: 19, HV: 0.0 Iteration: 20, HV: 0.040386589374653434 Iteration: 21, HV: 0.04137710053977417 Iteration: 22, HV: 0.04253929132347872 Iteration: 23, HV: 0.04148391340060306 Iteration: 24, HV: 0.04084955540330689
Expected HyperVolume Improvement. This is our current recommended algorithm for multi-objective optimization, when a reference point is already known.
ehvi_experiment = build_experiment()
ehvi_data = initialize_experiment(ehvi_experiment)
ehvi_hv_list = []
ehvi_model = None
for i in range(N_BATCH):
ehvi_model = get_MOO_EHVI(
experiment=ehvi_experiment,
data=ehvi_data,
)
generator_run = ehvi_model.gen(1)
trial = ehvi_experiment.new_trial(generator_run=generator_run)
trial.run()
ehvi_data = Data.from_multiple_data([ehvi_data, trial.fetch_data()])
exp_df = exp_to_df(ehvi_experiment)
outcomes = np.array(exp_df[['a', 'b']], dtype=np.double)
try:
hv = observed_hypervolume(modelbridge=ehvi_model)
except:
hv = 0
print("Failed to compute hv")
ehvi_hv_list.append(hv)
print(f"Iteration: {i}, HV: {hv}")
ehvi_outcomes = np.array(exp_to_df(ehvi_experiment)[['a', 'b']], dtype=np.double)
Iteration: 0, HV: 0.0 Iteration: 1, HV: 0.0 Iteration: 2, HV: 0.013921266712674477 Iteration: 3, HV: 0.013685730160361067 Iteration: 4, HV: 0.007234451578111365 Iteration: 5, HV: 0.007369586278411647 Iteration: 6, HV: 0.007572070189111986 Iteration: 7, HV: 0.008010352442337538 Iteration: 8, HV: 0.12603573635880294 Iteration: 9, HV: 0.15981500227611992 Iteration: 10, HV: 0.18334679821415045 Iteration: 11, HV: 0.19520895837203991 Iteration: 12, HV: 0.20905040673425845 Iteration: 13, HV: 0.2254583943068609 Iteration: 14, HV: 0.2333155382602158 Iteration: 15, HV: 0.2450407486597727 Iteration: 16, HV: 0.25570907473943927 Iteration: 17, HV: 0.2627225709401917 Iteration: 18, HV: 0.27272292798648623 Iteration: 19, HV: 0.28421800207014314 Iteration: 20, HV: 0.29520460601902915 Iteration: 21, HV: 0.3019694837839276 Iteration: 22, HV: 0.3126137527317435 Iteration: 23, HV: 0.323066142261177 Iteration: 24, HV: 0.3291297927035098
The plotted points are samples from the fitted model's posterior, not observed samples.
frontier = compute_posterior_pareto_frontier(
experiment=ehvi_experiment,
data=ehvi_experiment.fetch_data(),
primary_objective=metric_b,
secondary_objective=metric_a,
absolute_metrics=["a", "b"],
num_points=20,
)
render(plot_pareto_frontier(frontier, CI_level=0.90))
This is a good alternative algorithm for multi-objective optimization when qEHVI runs too slowly or produces poor results.
parego_experiment = build_experiment()
parego_data = initialize_experiment(parego_experiment)
parego_hv_list = []
parego_model = None
for i in range(N_BATCH):
parego_model = get_MOO_PAREGO(
experiment=parego_experiment,
data=parego_data,
)
generator_run = parego_model.gen(1)
trial = parego_experiment.new_trial(generator_run=generator_run)
trial.run()
parego_data = Data.from_multiple_data([parego_data, trial.fetch_data()])
exp_df = exp_to_df(parego_experiment)
outcomes = np.array(exp_df[['a', 'b']], dtype=np.double)
try:
hv = observed_hypervolume(modelbridge=parego_model)
except:
hv = 0
print("Failed to compute hv")
parego_hv_list.append(hv)
print(f"Iteration: {i}, HV: {hv}")
parego_outcomes = np.array(exp_to_df(parego_experiment)[['a', 'b']], dtype=np.double)
Iteration: 0, HV: 0.0 Iteration: 1, HV: 0.0 Iteration: 2, HV: 0.01604318809469206 Iteration: 3, HV: 0.014127038626635046 Iteration: 4, HV: 0.012508846874611002 Iteration: 5, HV: 0.013264292308452474 Iteration: 6, HV: 0.18843496612811467 Iteration: 7, HV: 0.1947224283691514 Iteration: 8, HV: 0.1122510306051966 Iteration: 9, HV: 0.10854298883114527 Iteration: 10, HV: 0.11054729103496745 Iteration: 11, HV: 0.1139484579371268 Iteration: 12, HV: 0.1151621076249975 Iteration: 13, HV: 0.15633744893146295 Iteration: 14, HV: 0.1733336824860093 Iteration: 15, HV: 0.18101407307899775 Iteration: 16, HV: 0.18661475573672134 Iteration: 17, HV: 0.19325410687008465 Iteration: 18, HV: 0.19946811774061746 Iteration: 19, HV: 0.20633372103247066 Iteration: 20, HV: 0.2098177700392207 Iteration: 21, HV: 0.21446521822251047 Iteration: 22, HV: 0.2097379388774188 Iteration: 23, HV: 0.2060723822386083 Iteration: 24, HV: 0.20481642985960175
The plotted points are samples from the fitted model's posterior, not observed samples.
frontier = compute_posterior_pareto_frontier(
experiment=parego_experiment,
data=parego_experiment.fetch_data(),
primary_objective=metric_b,
secondary_objective=metric_a,
absolute_metrics=["a", "b"],
num_points=20,
)
render(plot_pareto_frontier(frontier, CI_level=0.90))
To examine optimization process from another perspective, we plot the collected observations under each algorithm where the color corresponds to the BO iteration at which the point was collected. The plot on the right for $q$EHVI shows that the $q$EHVI quickly identifies the Pareto front and most of its evaluations are very close to the Pareto front. $q$ParEGO also identifies has many observations close to the Pareto front, but relies on optimizing random scalarizations, which is a less principled way of optimizing the Pareto front compared to $q$EHVI, which explicitly attempts focuses on improving the Pareto front. Sobol generates random points and has few points close to the Pareto front.
import numpy as np
from matplotlib import pyplot as plt
%matplotlib inline
from matplotlib.cm import ScalarMappable
fig, axes = plt.subplots(1, 3, figsize=(20,6))
algos = ["Sobol", "parEGO", "EHVI"]
outcomes_list = [sobol_outcomes, parego_outcomes, ehvi_outcomes]
cm = plt.cm.get_cmap('viridis')
BATCH_SIZE = 1
n_results = N_BATCH*BATCH_SIZE + N_INIT
batch_number = torch.cat([torch.zeros(N_INIT), torch.arange(1, N_BATCH+1).repeat(BATCH_SIZE, 1).t().reshape(-1)]).numpy()
for i, train_obj in enumerate(outcomes_list):
x = i
sc = axes[x].scatter(train_obj[:n_results, 0], train_obj[:n_results,1], c=batch_number[:n_results], alpha=0.8)
axes[x].set_title(algos[i])
axes[x].set_xlabel("Objective 1")
axes[x].set_xlim(-150, 5)
axes[x].set_ylim(-15, 0)
axes[0].set_ylabel("Objective 2")
norm = plt.Normalize(batch_number.min(), batch_number.max())
sm = ScalarMappable(norm=norm, cmap=cm)
sm.set_array([])
fig.subplots_adjust(right=0.9)
cbar_ax = fig.add_axes([0.93, 0.15, 0.01, 0.7])
cbar = fig.colorbar(sm, cax=cbar_ax)
cbar.ax.set_title("Iteration")
Text(0.5, 1.0, 'Iteration')
The hypervolume of the space dominated by points that dominate the reference point.
The plot below shows a common metric of multi-objective optimization performance when the true Pareto frontier is known: the log difference between the hypervolume of the true Pareto front and the hypervolume of the approximate Pareto front identified by each algorithm. The log hypervolume difference is plotted at each step of the optimization for each of the algorithms.
The plot show that $q$EHVI vastly outperforms $q$ParEGO which outperforms the Sobol baseline.
import numpy as np
from matplotlib import pyplot as plt
%matplotlib inline
iters = np.arange(1, N_BATCH + 1)
log_hv_difference_sobol = np.log10(branin_currin.max_hv - np.asarray(sobol_hv_list))[:N_BATCH + 1]
log_hv_difference_parego = np.log10(branin_currin.max_hv - np.asarray(parego_hv_list))[:N_BATCH + 1]
log_hv_difference_ehvi = np.log10(branin_currin.max_hv - np.asarray(ehvi_hv_list))[:N_BATCH + 1]
fig, ax = plt.subplots(1, 1, figsize=(8, 6))
ax.plot(iters, log_hv_difference_sobol, label="Sobol", linewidth=1.5)
ax.plot(iters, log_hv_difference_parego, label="qParEGO", linewidth=1.5)
ax.plot(iters, log_hv_difference_ehvi, label="qEHVI", linewidth=1.5)
ax.set(xlabel='number of observations (beyond initial points)', ylabel='Log Hypervolume Difference')
ax.legend(loc="lower right")
<matplotlib.legend.Legend at 0x7f5cd4186650>
Total runtime of script: 3 minutes, 52.96 seconds.